-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix sync_dist for tpus #6950
Fix sync_dist for tpus #6950
Conversation
Codecov Report
@@ Coverage Diff @@
## master #6950 +/- ##
=======================================
- Coverage 92% 86% -5%
=======================================
Files 194 194
Lines 12366 12368 +2
=======================================
- Hits 11350 10697 -653
- Misses 1016 1671 +655 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM !
@@ -318,3 +313,9 @@ def register_ddp_comm_hook( | |||
state=ddp_comm_state, | |||
hook=ddp_comm_hook, | |||
) | |||
|
|||
|
|||
def tpu_distributed(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neat !
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
(cherry picked from commit 1b3e4f9)
(cherry picked from commit 1b3e4f9)
(cherry picked from commit 1b3e4f9)
(cherry picked from commit 1b3e4f9)
What does this PR do?
Fixes #6922
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃